Planning and Navigation Assistance using Two-dimensionally 
Adapted Generic and Detected Patient Data 



RELATED APPLICATION DATA 

This application claims priority of U.S. Provisional Application No. 
60/494,935 filed on August 13, 2003, which is incorporated herein by reference in 
its entirety. 

FIELD OF THE INVENTION 

The present invention relates to a method and device for 
computer-assisted medical navigation and/or pre-operative treatment planning. In 
particular, it relates to a method and device for planning and navigation 
assistance using two-dimensionally adapted and detected patient data. 

BACKGROUND OF THE INVENTION 

Computer-assisted, stereotactic systems, which work with the aid of body 
structure data obtained from tomographic detection systems and with the 
assistance of x-ray images produced in situ are known, for example, from U.S. 
Patent Nos. 4,791 ,934 and 5,799,055. X-ray imaging used to assist in operations 
is discussed in U.S. Patent Nos. 5,967,982, 5,772,594 and 5,784,431. 

Referencing the current position of the structures to be operated on by 
means of pre-operatively produced three-dimensional patient data and x-ray 
images produced in situ (fluoro-CT matching) is introduced here as a non-contact 
method, which is therefore suitable for minimally invasive surgery. 

Where accurate medical navigation is to be provided, the prior art still 
works with the aid of body structure data originating from tomographic detection 
systems, such as, for example, computer tomography devices or nuclear spin 
tomography devices. The patient to be treated is thus positionally registered in 
situ with respect to the image data determined beforehand. Operating 
instruments are then virtually displayed in the same relation to the image data as 
to the actual patient, to make the body structure data or, if possible, also x-ray 
image data useful to the surgeon in the operating room. 



Disadvantages of such methods, in which tomographs (CT, MR) or x-ray 
images are produced, especially for navigating within the framework of treatment, 
include the radiation load on the patient and the high costs, since such devices 
are very expensive both to purchase and to maintain and operate. 

5 Attempts have been made to develop systems that may be employed 

without body structure data separately detected beforehand, for example, based 
on statistical models of image data sets for body structures. However, such 
systems lack the required accuracy for the respective patient to be treated. 

DE 100 37 491 A1 and WO 99/59106 describe methods for providing 3D 

10 information with the aid of fluoro-images. The starting point in all methods is 
recording transillumination images of the patient or of the desired structure. A 
localization system is used to obtain spatial information using the images. DE 
100 37 491 A1 initially uses two fluoro-images, from which to reconstruct a rough 
3D model. Other images from different angles are used to specify the model 

is more precisely. In accordance with WO 99/59106, at least three fluoro-images of 
the patient are generally made. These are anterior-posterior, lateral and 
anterior-posterior with the head inclined backwards. In addition to the 
transillumination images, photographs of the patient are also used. In this prior 
art, the model is painstakingly adapted in three-dimensional space. 

20 

SUMMARY OF THE INVENTION 

In accordance with one aspect of the invention, a method for 
computer-assisted medical navigation or pre-operative treatment planning can 
include detecting a position of a patient or a part of a patient's body and detecting 
25 positions of medical treatment devices or treatment-assisting devices. The 
method can further include assigning the detected positions to body structure 
data, the body structure data being obtained from a three-dimensional generic 
model. 

30 BRIEF DESCRIPTION OF THE DRAWINGS 

These and further features of the present invention will be apparent with 
reference to the following description and drawings, wherein: 
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Fig. 1 is a flow diagram illustrating a method for registering and adapting a 
generic model in accordance with the invention; and 

Figs. 2 to 6 are exemplary screen shots of a computer application which 
assists the method in accordance with the invention. 

DETAILED DESCRIPTION OF THE INVENTION 

In accordance with one aspect of the invention, the current position of a 
patient or a part of a patient's body and the positions of medical treatment devices 
or treatment-assisting devices can be detected in such navigation methods by 
means of a position detection unit, and the detected positional data can be 
assigned to body structure data, in order to jointly use the body structure data in 
assignment with the positional data, within the context of assisting the treatment. 
Such a navigation system is described for example in DE 196 39 615 C2 and 
corresponding U.S. Patent No. 6,351,659, which is incorporated herein by 
reference in its entirety. 

In accordance with one aspect of the invention, the invention is directed to 
a method for computer-assisted medical navigation and/or pre-operative 
treatment planning. The method can avoid producing a separate image data set, 
which is cost-intensive and a burden to health, for navigation/treatment planning, 
while nonetheless providing sufficiently accurate navigation. 

In accordance with one aspect of the invention, in a computer-assisted 
medical navigation method and/or treatment planning method, body structure 
data is used. The body structure data can be obtained based on a generic 
model, which has been adapted by linking it with patient-characteristic detection 
data, such as pre-operatively or intra-operatively obtained x-ray image data. 

If the x-ray image data includes more than one image from different 
directions of projection, then these two-dimensional summation images can be in 
a known or calculable navigational relationship to each other. 

On the basis of this known relationship, a three-dimensional generic model 
is superimposed onto the patient-specific x-ray images, and a projection of this 
model is adapted to the respective x-ray images. The model can be adapted in 



two-dimensional space, unlike other described methods, which employ 
two-dimensional patient-specific data to create a three-dimensional model. 

When adapting in this way, the projection of anatomic landmarks or 
geometries, such as the edges of bones or typical osseous borders and 

5 anatomical structures (for example, the processus transversus and spinosus of a 
vertebra, or the projected course of the pedical in a vertebra model), are 
automatically or manually identified. Projecting the stored model structures is 
then adapted to these two-dimensionally identified landmarks. 

This adaptation requires a guideline for transforming the model onto the 

10 patient-specific data, which also enables the information stored in the model, for 
example, a CT or MR data set, to be appropriately modified, such that a "virtual" 
data set of the patient, including tomographic images, can be used for navigation. 

By calculating the projection, this data set can be displayed as digital 
reconstructed radiography (DRR) and compared with the patient-specific data in 

15 order to automatically or manually verify the model. 

In one embodiment, two transillumination images are sufficient. It is 
advisable to record these perpendicular to each other, such as, for example, in 
the case of anterior-posterior and lateral images, though other angles are not a 
problem. Other transillumination images can, however, also be incorporated in 

20 order to make the model more precise. 

In one embodiment, the generic model is pre-defined. It can contain 
information on the deforming models used and their interrelationships, as well as 
landmarks or other points of interest, which can be used to deform the model. 
The model can include either a surface model or a tomographic data model or a 

25 combination of both. One advantage of the surface model is that it is simple and 
quick to deform, which makes it particularly suitable for real-time applications. 
Tomographic data models are required for specific visualization and provide 
extended information on the volume of a structure. 

In order to use the three-dimensional model in conjunction with the 

30 transillumination images, it has to be adapted ("morphed") to the information of 
the images. In one embodiment, the landmarks and other points of interest, 
which are used for deforming, are projected onto the two-dimensional 
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transillumination images. The deformation of the model is defined by adjusting 
the predetermined landmarks to the contour of the desired structure. 

The landmarks do not have to be identified in more than one 
transillumination image. The prerequisite that at least two images are required in 

5 order to establish the position of the landmarks, as is required in the prior art, 
does not apply to the method described herein. It is sufficient if the landmarks 
are placed in just one of the images. Contrary to the conventional method as set 
forth in DE 100 37 491 A1 , a rough model is not produced from the projections, 
since the model in the method in accordance with the invention is predefined and, 

10 therefore, can be optimized for precise displaying. The number of landmarks to 
be identified is also substantially lower than in the method as set forth in WO 
99/59106. For the example of the spine, at most, 20 landmarks are required. 

The tomographic image data can be predefined together with the model, 
and further pre-operative tomographic imaging does not have to performed. 

15 Since the model used represents a predefined reference of the structure to 

be visualized, it only has to be adapted to the transillumination images. One 
advantage for the surgeon is that he obtains a precise 3D representation of the 
desired structure, irrespective of the number of transillumination images and 
without having to fall back on elaborate and costly tomographic image data (MR). 

20 In methods such as those described in the specifications of DE 100 37 491 

A1 and WO 99/59106, the precision in registering the desired structure is 
dependent on placing the landmarks and/or identifying the contour in the 
transillumination image. In the method in accordance with one embodiment of 
the invention, this error can still be minimized. To this end, the transillumination 

25 data predefined with the generic model are used. The deforming guideline, 

generated from the information of the transillumination images, is then applied to 
the tomographic data. A tomographic image data set, adapted to the patient, can 
be obtained. Using a fluoro-to-CT matching algorithm, the deformed tomographic 
image data set can then additionally be registered. This minimizes inaccuracies 

30 that can arise from manually or automatically placing the landmarks. With the aid 
of this step, the precision of the method can be substantially increased and 
reliability can also be improved. The precision of the fluoro-to-CT matching 
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method has been demonstrated in a number of studies, and the results of this 
algorithm are easily verified. 

When "navigation" is mentioned in the following, this term is intended in 
principle to also include pre-operative treatment planning in the context of which a 

5 surgeon determines an ideal approach for a treatment beforehand, for example, 
manually determines and fixes the ideal position of an implant. 

The advantages of the present invention are based, among other things, 
on the fact that by using a generic model adapted to the patient, it is no longer 
necessary to produce a separate data set for the body structure, for the treatment 

10 for which medical navigation is to be provided. This spares the patient the 

radiation load as well as saves the costs of producing the data set (for example, 
by tomography). In addition, linking the generic body structure data with 
patient-characteristic detection data provides a data set which enables highly 
accurate medical navigation. The generic model, which can be a kind of universal 

15 model for the body structure in question, for which all the relevant data is 

available, does not include data tailored specifically to the patient in question, but 
does include sufficient anatomical and/or body structure data to be able to 
provide a sufficiently accurate basis for medical navigation, once it has been 
adapted with the aid of patient-characteristic detection data. 

20 It is possible within the framework of the present invention to provide the 

body structure data in the form of an image data set, such as a tomographic 
imaging data set. 

In this way, separate image detection data sets are not generated, as in 
methods in accordance with the prior art. Rather, the generic model itself is 

25 already provided in the form of an image data set, which can then be adapted to 
the respective patient, to obtain an image data set that is valid for the patient. 
The image data set is adapted by way of superimposing, for example, 
patient-specific x-ray images, which represent a two-dimensional summation 
image from a defined direction of projection, and projecting the three-dimensional 

30 generic model onto said summation image. 

The projection of the model onto the patient-specific data is adapted using 
knowledge of the three-dimensional, model-specific interdependence between 
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anatomical landmarks and structures. Thus, manual or automatic shaping 
between the model and the patient data can, depending on the structure, 
correspond to shifting, deforming or rotating the model, or to a combination of 
these. 

5 The deforming, shifting or rotating guideline thus obtained for the model 

can then be applied to the information stored in the model, to generate a 
three-dimensional image data set or to deform an already existing image data set 
with the aid of the guideline. 

This image data set can then be employed just like one produced 

10 cost-intensively and with pre-operative radiation load for the patient. It is 

conceivable, for example, to use a generic model comprising a typical or average 
body structure, for example, a simple model representation of a vertebra, 
humerus, radius or ulna, femur, tibia or pelvic bone, or another osseous body 
structure. 

15 The generic model can also include a statistical model of the body 

structure, such as one based on statistical evaluations of an indefinite number of 
image data sets, for example, of actual vertebra image data sets. 

Furthermore, the possibility exists of providing the generic model directly 
as a kind of model package for a multitude of body structures of the same type. 

20 In this case, it is possible when adapting the model to isolate from the multitude of 
models in the package the one that best matches the patient-characteristic 
detection data, such that.the model only has to be slightly adapted with computer 
assistance. 

Within the framework of the present invention, the generic model can 
25 include a two- or three-dimensional data set of a body structure, also a geometric 
model. In other words, the generic model can include both three-dimensional 
data (for example, a vertebra model) and two-dimensional data (for example, 
virtual x-ray images) or also a model in the form of geometric data. This data can, 
for example, be angles and/or trajectory information, which can be displayed for 
30 the physician and indicate to him the ideal position of an implant. 

Various types of patient-characteristic data are outlined in the following, 
such as can be used for adapting the generic model. It is also always possible to 
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employ combinations of such data, referred to as diagnostic data in the following, 
to this end. 

The patient-characteristic data can be x-ray data, from x-ray images 
produced before or during treatment, such as bi-planar or multi-planar x-ray 
images. An example of this is when x-ray images of the patient are already 
available, which were produced within the context of previous examinations. Data 
about body structures from these "old" x-ray images is particularly suitable if 
deviations of form with respect to the generic model are to be calculated in. 

It is, however, also possible, even during treatment, to produce individual 
x-ray images of the patient and to include this information in adapting the generic 
model. The advantage as compared to conventional "x-ray navigation" is then 
that it is not necessary to produce a large multitude of x-ray images, such as are 
used in navigation based on x-ray images. It is sufficient for adapting the generic 
model to produce just one or very few x-ray images, which, moreover, can be 
restricted to a very small section of the body. This significantly reduces the 
radiation load as compared to conventional x-ray navigation. 

The above applies in the same way to computer tomography or nuclear 
spin tomography image data. Data may be used, which is derived from 
tomographic images produced much earlier, but whose information is sufficient 
for suitably adapting the generic model. 

Moreover, the diagnostic image data can also be digitally reconstructed 
x-ray image data (DRRs = digitally reconstructed radiographs), which can, for 
example, be produced from tomographic image data sets already available, 
without the patient again having to be subjected to x-ray imaging. 

It is, however, not absolutely necessary to use complicated, 
patient-characteristic detection data and/or diagnostic data in this way, to be able 
to adapt the generic model sufficiently. It can be sufficient to use acquired 
point-positional information of the patient's body structure, such as natural or 
artificial landmarks. The patient-characteristic diagnostic data can then, for 
example, be the distance between two landmarks (for example, apophyses), 
which alone can give sufficient information about how the generic model should 



be restructured. Similarly, data on size, weight or lengths of the body section or 
of one or more limbs of the patient can be used as a basis for this. 

The generic model can be adapted within the context of the invention using 
one or more of the following methods: manually adapting with the assistance of 

5 image representation, such as by offsetting points and landmarks or by shifting, 
rotating, stretching or compressing the generic model on a screen output by 
means of user-interface means; automatic image fusion methods, for example, 
based on automatically identifying particular anatomical features; and registering 
and/or fusing image data of the generic model, such as digitally reconstructed 

10 x-ray images, and the same from computer tomography or nuclear spin 
tomography image data sets. 

The generic model can thus be fused using diagnostic methods either 
automatically, for example, by automatically identifying particular anatomical 
features critical for fusion, or manually, for example, by shifting, rotating and/or 

15 stretching. When the generic model is fused with actual patient information by 
acquiring an indefinite quantity of point information on the patient (landmarks), it is 
possible to use a so-called surface-matching method, i.e., a computer-assisted 
image adapting method, to fuse the image data. From the various methods 
described above, capturing the diagnostic data and adapting the generic model 

20 are combined in accordance with one embodiment of the invention, such that 
alongside the diagnostic data (for example, x-ray images acquired 
intra-operatively), additional points on the patient are also recorded, in the form of 
landmarks or randomly acquired points, and used to detect and adjust the 
position of the model or its form even more accurately, so as to enable more 

25 accurate navigation. 

A hip-thigh arrangement (pelvis/femur) can also be registered by means of 
registered x-ray images and a generic model. The hip could then be registered, 
for example, by assigning landmarks between the generic model and one or more 
x-ray images (fluoroscopy, for example at an angle of 30°) by mathematical 

30 coupling. 

Generally speaking, the positional data obtained while determining the 
patient-characteristic detection data, such as by acquiring landmark positions or 



-9- 



by x-ray imaging registered in the navigation system, can be used to register the 
adapted body structure data in the navigation system and to visually display or 
introduce treatment devices and/or treatment-assisting devices in their registration 
to the adapted body structure. In other words, the step of capturing the diagnostic 
data is also simultaneously used in this way to register the patient and the 
adapted generic model for navigation. As long as the data of the model is fused 
with registered data, i.e., data which is clearly determined in the spatial position, 
such as registered fluoroscopy images of an x-ray navigation software, or the data 
of the model are registered with landmarks, or a combination of the two methods, 
these can be used for computer-assisted surgery and, for example, for minimally 
invasive operations in which instruments or implants are displayed in relation to a 
fused model. 

The method described herein can be used both to assist in surgery in 
which the surgeon is provided with navigating aids on screens, and within the 
context of radiotherapy and/or radiosurgery. Navigation can be based on optical 
tracking or on magnetic tracking. 

The present invention can include a program which, when running on a 
computer or loaded on a computer, causes the computer to perform one or more 
of the methods described above, and a computer program storage medium 
comprising such a program. 

In summary, it may further be stated with respect to the above invention 
that it reduces the radiation load and eliminates or at least minimizes the costs of 
tomographic imaging methods, and that it also has the advantage as compared to 
pure x-ray navigation that it enables the surgeon three-dimensional navigation 
and orientation. Further advantages lie in the fact that the steps resulting in the 
patient being registered are substantially less complicated and the patient can be 
registered using few manual steps. Costly diagnostic examinations are simplified, 
and acquiring points/landmarks on the patient for registering can also be made 
superfluous, if already calibrated data (for example, registered x-ray data) are 
used and additional useful information is obtained through fusion with the generic 
model (for example, converting a quantity of two-dimensional information into 
actual three-dimensional information). 
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It is thus possible in accordance with the invention to automatically display 
the ideal position of implants or instruments at little cost, such that surgery can be 
performed more quickly, more securely, and less invasively. 

With reference now to Fig. 1, a flow chart illustrating a method for 

5 registering and adapting a generic model is provided. It is to be appreciated that 
the method illustrated in Fig. 1 is described in greater detail above. The method 
can include recording one or more two-dimensional fluoro-images at step 10 and 
selecting a predefined, available generic model at step 12. Step 16 can include 
projecting the generic model on the two-dimensional images, adapting landmarks 

10 in the two-dimensional images, and applying the appropriate deformation to the 
generic model. Step 20 can include applying deformation to the tomographic 
data belonging to the generic mode. Step 24 can include converting tomographic 
data into DRRs and performing fluoro-to-CT matching. At step 28, a model and 
tomographic data, which are more exactly and/or precisely registered, results. 

is This makes surgical navigation possible using, for example, the tomographic data 
and a surface model. 

Figs. 2-6 show screen shots of a computer application, which assists the 
method described herein, in accordance with one embodiment of the invention. 
Screen shots for two-dimensionally superimposing x-ray images and projections 

20 from generic models may be recognized, including highlighted landmarks of the 
model, which are placed on the corresponding points of the patient x-ray images 
by the operator, so as to ascertain a suitable transformation. 

Fig. 2 shows two exemplary views of a hip model, while Figs. 3 and 4 show 
two exemplary views of a vertebra model, wherein only the landmarks are 

25 indicated. Fig. 5 shows exemplary views of a femur. 

Once the transformation guideline is then ascertained and the generic 
model adapted, fluoro-navigation can be performed as shown in Fig. 6, on the 
x-ray images (top) and assisted by adapted generic 3D image representations 
(bottom). 

30 Although the invention has been shown and described with respect to a 

certain preferred embodiment or embodiments, it is obvious that equivalent 
alterations and modifications will occur to others skilled in the art upon the reading 
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and understanding of this specification and the annexed drawings. In particular 
regard to the various functions performed by the above described elements 
(components, assemblies, devices, systems, compositions, etc.), the terms 
(including a reference to a "means") used to describe such elements are intended 
to correspond, unless otherwise indicated, to any element which performs the 
specified function of the described element (i.e., that is functionally equivalent), 
even though not structurally equivalent to the disclosed structure which performs 
the function in the herein illustrated exemplary embodiment or embodiments of 
the invention. In addition, while a particular feature of the invention may have 
been described above with respect to only one or more of several illustrated 
embodiments, such feature may be combined with one or more other features of 
the other embodiments, as may be desired and advantageous for any given or 
particular application. 



